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The first search in pp coUisions at ^/s — 1.96 TeV for the production of neutral Higgs bosons in 
association witfi bottom quarks and decaying in two tan leptons is presented. The cross section for 
this process is enhanced in many extensions of the standard model (SM), such as its minimal super- 
symmetric extension (MSSM) at large tan/3. The data, corresponding to an integrated luminosity 
of 328 pb~^, were collected with the DO detector at the Fermilab Tevatron Collider. An upper limit 
is set on the production cross section of neutral Higgs bosons in the mass range of 90 to 150 GeV, 
and this limit is used to exclude part of the MSSM parameter space. 

PACS numbers: 14.80.Cp, 12.60.Pr, 12.60.Jv, 13.85.Rm 



In the minimal supersymmetric extension of the stan- dard model (MSSM), the Higgs sector consists of five 
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physical Higgs bosons: two neutral scalars, h and H 
(with mh < rnu by convention), one neutral pseudo- 
scalar, A, and a charged pair, H^. At leading order 
(LO), the coupling of the neutral Higgs bosons to down- 
type quarks is proportional to tan/3, where tan/3 is the 
ratio of the vacuum expectation values of the two Higgs 
doublets. The production cross section of a neutral Higgs 
boson in association with a down-type quark, such as the 
h quark, is therefore proportional to tan^/3 (at LO). Thus, 
the b(j) {(f) = h, H, A) production mechanism provides a 
natural mode to search for a neutral Higgs boson at high 
tan/3 in the MSSM 0. 

In most of the MSSM parameter phase space, the neu- 
tral scalar Higgs bosons h and H decay ~90% of the 
time into a pair of b quarks, and ^10% of the time into 
a pair of tau leptons. The neutral pseudoscalar A decays 
into bb or t^t~ in all of the parameter space, with simi- 
lar branching ratios (~90% and ~10%, respectively). In 
this Letter, we present a search for the production of a 
neutral Higgs boson in association with a b quark, with 
the subsequent decay of the Higgs boson into two tau 
leptons, using data collected by the DO experiment in pp 
collisions at ^/s = 1.96 TeV at the Fermilab Tevatron col- 
lider. We perform the analysis using the final state where 
one tau decays leptonically into a muon (t ~f (iv^i't), 
and the other tau decays hadronically into a narrow jet 
(r ThVr, where Th denotes the hadronic tau jet). 

The b(j) — > bT^T~ search channel is complementary to 
the b4> bbb Q and the inclusive (j) t^t~ Q searches. 
The r+T~ decay mode of the Higgs boson is less sensitive 
than the bb decay to the large supersymmetric radiative 
corrections on the production cross section and decay 
width [l|. Experimentally, the 6r+T^ channel presents 
a clean signature which does not suffer from the large 
heavy- flavor multi-jet background of the bbb channel, and 
is less affected by the Z t^t^ background than the 
inclusive (j) — > r+r" channel. 

The DO detector Q consists of a central tracking sys- 
tem, comprising a silicon microstrip tracker and a cen- 
tral fiber tracker, both within a 2 T solenoidal magnet; a 
liquid-argon and uranium calorimeter, divided into a cen- 
tral calorimeter and two end calorimeters; and a muon 
system, consisting of three layers of tracking detectors 
and scintillation trigger counters. 

This analysis considers data collected by the DO exper- 
iment between August 2002 and June 2004. Two single- 
muon triggers are used, requiring a muon with transverse 
momentum {px) greater than either 3 or 5 GeV and a 
track with pT > 10 GeV. The total integrated luminosity 
for the selected triggers is 328 ± 20 pb~^ ^^i]- 

Signal events are simulated using the process pp 
b(j) bT~^T~ in PYTHIA Q, where one of the tau leptons 
is forced to decay leptonically into a muon and the sec- 
ond tau is free to decay to all allowed modes; the b quark 
is generated with p-r > 15 GeV and |ry| < 2.5, where 
T] = — ln[tan(6'/2)] is the pseudorapidity and 6 is the 



polar angle relative to the proton beam direction. Back- 
ground processes such as tt, W+iets and WW production 
are simulated using ALPGEN Q interfaced with pythia 
for showering and fragmentation. Additional pp inter- 
actions are modeled with pythia according to a Pois- 
son distribution with mean of 0.4 events, which corre- 
sponds to the expected average multiplicity in the data. 
The simulated events are processed through a GEANT- 
based Q simulation of the DO detector and reconstructed 
with the same software as the collider data. They are also 
weighted on an event-by-event basis by the trigger effi- 
ciency parametrization measured in the data. The trigger 
efficiency, estimated on the simulated signal sample after 
selecting /xr/i pairs, is (62 ± 1)%. 

There are three types of physics objects used in this 
analysis: muons, hadronic taus, and jets. All selected 
objects are required to be associated with the same pri- 
mary vertex within 1 cm along the beam direction. 

Muons are reconstructed from patterns of hits in the 
muon detectors matched to isolated central tracks, and 
are required to have pr > 12 GeV. 

Hadronically decaying taus are characterized by a nar- 
row isolated jet with low track multiplicity. We distin- 
guish three tau types: (1) a single track with energy de- 
posited in the hadronic calorimeter, (2) a single track 
with energy deposited both in the hadronic and electro- 
magnetic calorimeters, and (3) three tracks with corre- 
sponding energy deposited in the calorimeter. 

After an initial selection of tau candidates based on the 
transverse energy (Et) of the calorimeter cluster, sum of 
the track transverse momenta, and isolation and width 
of the associated calorimeter energy deposits, the candi- 
dates are further discriminated against jets using a neural 
network (NN) which has been trained separately for each 
tau type For types 1 and 2, tau candidates are re- 
quired to have a NN output greater than 0.8. For type 
3 tau candidates, because of the larger multijet back- 
ground, the NN selection is tightened to 0.98. The aver- 
age tau identification efficiency in signal events is ~62%. 

Jets are reconstructed from clusters of energy in the 
calorimeter using the DO Run II midpoint cone algorithm 
with a radius of 0.5 [l3|. Jet energies are corrected to the 
particle level. Events are required to have at least one 
jet identified as originating from a b quark (b tagged) 
and with px > 15 GeV and \r]\ < 2.5. Jets are b tagged 
using an algorithm that computes the probability that 
the jet originated from a b hadron, based on the impact 
parameter of the tracks associated with the jet [11|. For 
a jet oipT = 20 GeV and jryj < 2.5, as is typical for signal 
events, the 6-tagging efficiency measured in data is ^-^40%, 
whereas the probability to tag a light-fiavor jet is ^1%. 
A parameterization of the 6-tagging efficiency measured 
in data is applied to each simulated jet, according to its 
Pt, rj and flavor. 

Main backgrounds to the b(j) — > for+r^ — > bfiT^ process 
are multijet, Z+]ets and tt production. Smaller back- 
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ground contributions originate from W+jets and WW 
production. The multijet and Z+jets backgrounds are 
estimated from the data, whereas all other backgrounds 
are estimated from the Monte Carlo (MC) simulation. 

A multijet background event typically consists of two 
or more jets, with one jet misidentified as a hadronic tau, 
a real or misidentified b jet, and a muon from a heavy- 
flavor decay that appears isolated. Since the charge 
of the muon is not correlated with the charge of the 
hadronic tau candidate, the multijet background tends 
to have equal amounts of opposite-sign (OS) and same- 
sign (SS) ^Tfi pairs. In contrast, the signal should contain 
only opposite-sign /iT/i pairs coming from the Higgs de- 
cay. Thus, we require that the reconstructed muon and 
hadronic tau have opposite charges. The multijet back- 
ground in the OS sample is estimated from the SS events 
in the data as follows: first, the SS yield is corrected for 
non- multijet backgrounds by subtracting these based on 
MC estimates; second, the corrected SS multijet yield 
is multiplied by the probability of a jet to be misiden- 
tified as a hadronic tau; third, a correction is applied 
to account for a small asymmetry observed in OS and 
SS multijet control samples; finally, the probability of a 
multijet event to have at least one 6-tagged jet is applied. 

The production oi a Z boson in association with jets 
contributes as a background via Z t~^t~ —^ fiTh and 
Z — !■ fJ-'^ n~ decays, and where one of the jets is a real or 
misidentified b jet. In the case of Z l-i-^ l^ i one of the 
muons is misidentified as a hadronic tau. The contribu- 
tion from both real and misidentified b jet backgrounds, 
in either Z decay channel, is estimated by measuring the 
fraction of 6-tagged events in Z ^ data, found to 

be (2.5±0.4)%, and multiplying it by the estimated num- 
ber of Z(— > ^r^)-l-jets events in data before b tagging. 

After b tagging, tt production is the dominant back- 
ground. Such events are characterized by having higher 
Pt objects than those in signal events. Therefore, in or- 
der to reduce the background, we use a neural network 
(KNN) which exploits kinematic differences between sig- 
nal and background, based on four variables: the sum 
of the transverse momenta of all jets in the event (ex- 
cluding the tau jet), the missing transverse energy 
(constructed from calorimeter cells and the momenta of 
muons, and corrected for the energy response of taus and 
jets), the jet multiplicity, and the azimuthal angular sep- 
aration between the muon and the tau jet. The neural 
network training is performed using a background MC 
sample of ti events where both W bosons decay lepton- 
ically {tt jiTh) and a signal MC sample consisting of 
b(j) — *■ bT~^T^ bfiTh events with a mixture of different 
Higgs masses. In both samples, the events used passed all 
selection criteria except b tagging. The KNN selection is 
optimized separately for each tau type. Events with type 
1 and 3 taus have low ti background and do not benefit 
from a KNN selection. Requiring a KNN output greater 
than 0.4 has a signal efhciency of ~95% and is found 



to be optimal for events with type 2 taus. The amount 
of tt background remaining after the KNN selection is 
estimated from MC. 

Systematic uncertainties affecting both signal and 
background predictions based on MC are: integrated lu- 
minosity (signal: 6%, background: <1%) [3|; trigger ef- 
ficiency (1.1%); tau identification (signal: 3-9%, back- 
ground: <0.4%); tau energy scale (10%); jet identifica- 
tion (signal: 6-9%, background: <7%); jet energy scale 
(signal: 7-10%, background: <4%); 6-jet identification 
(signal: 5%, background: <2%); and uncertainties on 
the signal (10%) and tt, W+iets (20-30%) and WW the- 
oretical cross sections. For backgrounds derived from 
data, the systematic uncertainties result from the lim- 
ited statistics of the control data samples. 

The estimated number of events from the various back- 
grounds and the observed number of events in the data 
for the three tau types are presented in Table HI Also 
shown are the signal acceptance and the number of ex- 
pected signal events for a Higgs mass = 120 GeV 
and tan/3 = 80. The visible mass M^is distributions, 
constructed from the four-vector momenta of the muon, 
hadronic tau, and missing momentum [3|, for the data 
and SM prediction are shown in Fig. [1] No visible excess 
over the SM prediction is observed in the data. 

TABLE I: Expected number of events for backgrounds, num- 
ber of observed events in data, signal acceptance for events 
with at least one muon and expected number of signal events 
for — 120 GeV and tan/3 — 80, for each hadronic tau 
type. Quoted uncertainties represent statistical and system- 
atic added in quadrature. 





Type 1 


Type 


3 2 


Type 3 


Muhijet 


0.60 ±0.22 


0.48 ± 


0.14 


0.95 ±0.16 


Z+jets 


0.34 ±0.09 


1.50 ± 


0.27 


0.25 ±0.08 


tt 


0.28 ±0.06 


0.65 ± 


0.18 


0.21 ±0.05 


W+]ets 


0.009 ± 0.005 


0.073 ± 


0.036 


0.28 ±0.12 


WW 





0.014 ± 


0.004 





Total Background 


1.22 ±0.19 


2.71 ± 


0.33 


1.68 ±0.15 


Observed 





1 




2 


Signal Accept. (%) 


0.15 ±0.03 


0.87 ± 


0.14 


0.27 ±0.05 


Expected Signal 


0.68 ±0.15 


3.9 ± 


0.7 


1.2 ±0.2 



Upper limits on the production cross section times 
branching ratio are set using a modified frequentist ap- 
proach [12l | . In order to maximize the sensitivity, each tau 
type is treated as a separate channel and the kinematic 
differences between signal and background are exploited 
by using the M^is distribution in the limit calculation. In 
each channel, the M^is distribution is split into three bins: 
30-60, 60-85 and 85-180 GeV (see Fig. [J). The choice of 
bin size is driven by the available statistics in data to 
estimate the multijet background. Figure [2] shows the 
95% confidence level (C.L.) upper limits on the produc- 
tion cross section times branching ratio as a function of 
the Higgs mass. Despite the ~1:9 branching ratio of the 
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M„i3 [GeV] M„i^ [GeV] [GeV] 

FIG. 1: Visible mass distributions for each tau type. Histograms show the signal and various backgrounds, points show the 
data. The error bands indicate the total uncertainty on the background estimation. 



T+r^ to 66 Higgs decay modes, the upper limit on the 
b(j) production cross section obtained by this analysis is 
competitive with the corresponding one in the b(j) bbb 
channel 0, particularly at low AI^. 

Using the cross section limit for bcj) production, wc 
can exclude regions of (m^, tan/3) parameter space in 
the MSSM. Beyond LO, the masses and couplings of the 
Higgs bosons in the MSSM depend (through radiative 
corrections) on additional SUSY parameters, besides uia 
and tan/3. Thus, we derive limits on tan/3 as a function of 
niA in two specific, commonly used scenarios (assuming 
a CP-conserving Higgs sector): the m™"^ scenario and 
the no- mixing scenario The production cross sec- 
tions, widths and branching ratios for the Higgs bosons 
are calculated over the mass range 90-150 GeV using the 
MCFM and FEYNHIGGS programs lEIiil- Since at large 
tan/3 the A boson is nearly degenerate in mass with ei- 
ther the h or the H boson, their production cross sections 
are added. As shown in Fig. [2l this analysis excludes a 
large portion of the MSSM parameter space. For nega- 
tive values of the Higgsino mass parameter fj,, the r+r^ 
decay mode explored here has comparable sensitivity to 
the bb decay mode For positive values of /i, how- 
ever, the r+r~ mode is superior to the bb mode, as it 
does not suffer from the effect of the large supersymmet- 
ric radiative corrections to the Higgs production cross 
section and decay width Compared to the inclusive 
— > r+r" channel for the same integrated luminos- 
ity the b(j) — > bT~^T~ channel offers increased sensitivity 
in the low region, as it does not suffer from the large 
Z T'^T~ background. 

In summary, we have presented results from a search 
for b(j) —^ br^T^ production, resulting in significant por- 
tions of the MSSM parameter space being excluded in 
two specific scenarios. This analysis is found to be both 



competitive and complementary to other searches in the 
b(f) bbb and inclusive cj) t^t~ channels, hence con- 
tributing to the overall sensitivity at the Tevatron. 
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